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ABSTRACT 



State-level concerns for postsecondary educ 
discussed as part of an inservice education conference, Att 
directed to the purposes of a performance approach, distinc 
between performance measures and other kinds of measures us 
assessment in higher education, concerns about reaching con 
performance criteria within a state versus the capacity for 
control being exercised, establishing the capacity to measu 
performance outcomes, the use of performance criterion in a 
evaluation process, and relating the performance measures t 
budgeting function. There appears to be no strong agreement 
conference participants on why performance measures are nec 
desirable and no clear evidence as to their consequences, T 
to be a consensus that questions of effectiveness and perfo 
would inevitably have to be faced regardless of thsir inten 
purpose. In establishing the capacity to measure performanc 
outcomes, a number of issues are highlighted: the content o 
particular performance criterion needs to be carefully deli 
issue of debate is whether performance measures should be s 
for programs or for institutions; and a major difficulty in 
performance indicators is the evaluation of other changes o 
in higher education that could affect the outcome indicator 
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INTRODUCTION 



My original thought in summarizing this conference was to use what 
we had learned about performance evaluation and budgeting to assess this 
conference and the organizers' preparation for it. It was at that point 
that I recognized that there were two basic approaches which we had dis- 
cussed. First is the "Complex-Rational" approach in which one begins by 
outlining the context of the conference, stating the purposes, forming 
performance criteria, reviewing how the program was carried out, design- 
ing a measurement system for data collection and analysis, and the devel- 
oping of a mechanism for transmitting that information into future con- 
ference planning. After giving that some thought, I concluded it was 
potentially too controversial; I did not want to take the risk of threat- 
ening anyone before my expenses were paid. The second approach is the 
"simplest possible measure approach". In this approach one accepts the 
one measurable criteria that we all agree on and then utilizes it. The 
measure that I would suggest is the efficacy of each participants ar- 
gument to his or her supervisor in order to convince them that in the 
Hotel del Coronado in San Diege by the Sea they A) worked, B) accomp- 
lished something worthwhile, and C) should still be reimbursed for that 
effort. I strongly suggest that Ken Fischer, obtain a transcript of 
the argument used by each participant, it may tell us a great deal about 
whether performance measures work and about how people are going to use 
performance evaluation to their own benefit. Besides, if it works, it 
might make a great argument for continued funding for this effort. 

v 

THE CONFERENCE: A CONTEXT 

This particular conference, as most of you know, was an outgrowth of 
or follow-up to a PECA conference in San Francisco in July, 1975 on 
"State Funding of Postsecondary Education: Incentives for Improvement". 
That conference featured Some yery blunt discussions of the conditions 
that were leading to a litany of things like: A) stagnation of insti- 
tutions of postsecondary education, B) excessive restraints on change, 
C) increased competitiveness for scarce resources, D) education as a 
sinking priority and E) the "Brown-Dukaki ethos" \in which the essen- 
tial character of government services are reevaluated. 

That conference identified as priorities for state-level action: 
1. The focus on a performance orientation; 2. The establishment of 
steps to preserve adaptability; 3. The development of cooperative means 
to improve utilization of resources; and 4. An emphasis on mechanisms 
for enhancing trust and rationality in relationships between states and 
institutions. The conference had concluded that a performance approach 
was worth serious consideration and, more specifically, that its con- 
nection to budgeting should be evaluated. This became the planning fo- 
cus for this conference on "Innovation, Outcomes and the State Budgeting 
Process". The program, however, has emphasized the .latter two points — 
outcomes and budget (See Appendix A — Final Program). 

The basic assumption underlying this conference, in my mind, is rep- 
resented by one of Piet Hein's Grooks . For those of you that have not 
seen his work, Hein is a Danish scientist, philosc. >ier, and intermittent 



poet who occasionally publishes collections of his Grooks . A Grook is 
a cartoon caricature accompanied by, a brief poem representing an eternal 
truth. The one that particularly comes to mind is the picture of a man 
on a tread mill, r unn ing and obviously very fatigued, with the caption: 
"He who tries to keep abreast is forever second best. 11 In the context of 
this particular conference the planners wi*re aware that institutional 
representatives, state legislators, executive budget officers, and state 
higher education agency staff all feel under substantial pressure regard- 
ing their interests in higher education and arc trying desperately to 
keep abreast if not to do even better. Unfortunately but accurately the 
secondary assuiaption appears to be that we may be often running in very 
different directions and therefore an "exploratory" dialogue about where 
we are heading with regard to performance measures and/or budgeting would 
be appropriate. (See Appendix B — Final list of participants and Appendix C 
— Ken Fischer's Summary of Attendees). 

The participants reasons for attending the conference were well sum- 
marized in the pre-conf erence interviews and by NEXUS staff and indicated 
a number of personal agendas which supported the planners assumptions. The 
first major concern was with the process by which states develop and im- 
plement outcome and performance measures whether for assessment or budgetary 
purposes. Your second major concern was to ask questions of each other. 
There was a great deal of interest in what's happening elsewhere? Why and 
how performance measures are used? What actually is the state cf the art 
in this area? 

A. What kinds of performance indicators or outcome measures are being used? 

B. What difficulties have different constituents encountered in obtaining 
useful information? What is useful to the different constituencies? 

C. How much information and' in what form should it be provided to various 
groups? and, 

D. How does one take into account outcomes that are not apparent or easily 
measurable? 

The conference fortnat, which emphasized autobiographical data from par- 
ticipants, allowed for considerable discussion, and provided for the 
presentation of examples of state-level activities underway, gave ample 
opportunity for addressing some of these concerns. The major statements by 
John Folger, Sid Micek and Howard Klebanoff addressed the pressures for, 
the initial NCHEMS studies of, and the the legislative perspective on out- 
comes of a performance orientation. Fred Pinkamfe interview with Howard 
Bowen, James Furman, and Eileen Anderson allowed institutional, state agency 
and state executive officers to identify their perspectives. Needless to 
say the legislative representatives in the audience were also well repre- 
sented in the discussion. John Folger f s paper started us with a series, \ 
Possible Implications and Questions, which the small groups addressed. The 
basis for more detailed discussions in small groups were well established 
(See Small group summaries attached) . 
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DOGMAS, DELUSIONS AND DILEMMAS 

The flow of the conference has led us to a conclusion in which there 
are extremes: From some dogmatic-: support for to some delusion with a 
performance or outcomes orientation- But inevitably the conference ends 
where it began: ..with issues sharpened into dilemmas, dilemmas for which 
each participant is now hopefully better prepared, but which will, no 
doubt, be resolved in the context of each state's unique style, governance 
arrangement, pattern of institution, and current set of problems. There 
are seven broad areas that hightlight some of these dilemmas, each of 
which is discussed briefly. 

A. The Problem Addressed: The Purposes of a Performance Approach . 

In John Folger t s initial paper and in many of the discussions that fol- 
lowed, it became apparent that there were a number of pressures or condi- • 
tions to explain why people are looking at a performance approach both for * 
assessment and potentially budgeting. In the .larger society , Folger noted 
the declining confidence and esteem that people hold for higher education, 
the "size of the higher education budget which has grown substantially to be- 
come a major proportion of most state's expenditures, and pressures of re- 
cession and inflation which are causing problems in resource allocation for 
state 9 s government as well as institutions of higher education. In state 
government he noted the extended concern for accountability reflected by the 
extensive growth of auditing staffs in the state executive offices and more 
recently in the legislative staff area. This was combined with a concern 
reported by several of our participants on the need for state level budgetary 
reform which indicated some greater affinity to make changes in that area. 
Within institutions of higher education this summer's PECA conference noted 
that nontraditional and innovative programs have felt stifled by the J input 
or effort oriented budget approaches currently in vogue. Similarly, insti- 
tutions faced with resources reductions have been forced to considered re- 
allocation and must consider discontinuance of ineffective as well as inef- 
fecient programs. Finally institutions are being subjected to increasing stu- 
dent consumer concerns, reinforced by student-based aid, ' in which students 
demand more information about the nature of the programs and institutions that 
they are entering. The pressures at these three levels all have supported 
to some extent a concern for a performance-oriented approach. This unfortun- 
ately has occurred when therr is a growing rift reported between legislators 
and higher educational officials as each becomes more cynical toward the 
other in a time of decline. -The inevitable consensus of the conference par- 
ticipants seemed to be that questions of effectiveness and performance would 
inevitably have to be faced regardless of their intended purpose or one's 
position on their desirability. 

The conference discussions themselves reflected considerable ambiguity 
and uncertainty regarding the consequences of the performance approach. There 
was support for the position this orientation might improve the performance 
of higher education if it were taken seriously and critically and if measures 
were applied with a substantial amount of diversity in different kinds of 
institutions, programs, and state situations. However, it could also be 
stifling if applied in more uniform ways. The same arguments, of course, 
apply to the consequences for diversity and innovativeness. There was some 
suggestion that, at least the attempt to provide performance indicators, might 
improve the legitimacy of higher education. It was also noted that the em- 
phasis from state legislators and state agencies, perhaps, a reflection of 
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their concern to retain legitimacy with their constitutencies by enabling 
them to point out what institutons were doing with the money they were 
appropriating or allocating. There was fear that the outcome oriented 
performance approaches might be too simplistic. However, the legislators 
pointed out the counter emphasis on the complexity of information derived 
to represent true performance led to their being inundated with data. There 
were also concerns, that outcome data might lead to further centralization 
by allowing higher education or state government agencies to dictate out- 
comes through the budget. But ways in which it might lead to greater in- 
stitutional autonomy by leaving management to the institutions were also 

suggested. 

't , .. 

The most crucial concerns about the purposes of a performance approach 
were: first, that performance or outcome questions inevitably and inex- 
tricably were value laden. Since different constituents not only would '■ N 
stress different performance criteria but may even interpret the same one 
differently, value conflicts would result. In this view the approach 
needed serious consideration rather than cursory review or perfunctory ac- 
ceptance. Second, there was a substantial concern that the performance 
approach be viewed as means rather than ends . This, in part, reflected 
the uncertainty about their consequences but may also have reflected an 
attempt to downplay or to avoid becoming overly confident about the gains 
that proponents claim for the imtilization of this kind of an approach. 
Opinions of participants themselves seemed to range from the optimistic 
extreme that this approach would allow higher education to get more state 
funds and perhaps even improve their autonomy to a pessimistic position 
that, regardless of the budgetary process and performance criteria* higher 
education in today's context would receive less funding and probably in- 
evitably lose autonomy. The middle ground position suggested that it was 
an approach which might protect quality in a time of retrenchment, might 
allow for a larger range of educational needs (Diversity, and Innovative) , 
and might provide a mechanism for reducing, some of the conflicts among 
participants in higher education policy making that we currently face. 

In the discussions of the purposes and consequences of a performance 
approach, conclusions were hard to reach. However, there was substantial 
agreement that they would be helpful in identifying problem* areas, that 
the development of them might be useful as a basis for dialogue among var- 
ious constituencies, that there was a need for higher education to contin- 
ue to try to make a better case for its share of state funds and that any 
performance approach should involve changes made with deliberate caution. 

B. The Semantics Syndrome . 

Perhaps one of the reasons for the general caution in discussing a 
performance approach to assessment and budgeting in higjher education is 
partially a semantics issue. Legislators to some extent see it as an- 
other fancy term developed by institutions to disguise the fact that no- 
thing has changed while institutions see this as just another "faddish" 
approach being instigated by the state. Regardless of one's perspective, 
it is also easy to see the term "outcomes" as just another example of 
"systems gobbledy-gook". Therefore, it is helpful to make some distinc- 
tions about what we really mean. 
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First, it is helpful to 'distinguish performance measures from other 
kinds of measures used for assessment in higher education. "Performance 
measures" generally refer' to the outcomes of higher educational institu- 
tions or the impacts they have on the students, the state, or the larger 
social institutions which they serve. Measures of performance can be quan- 
titative or qualitiative. They may be oriented toward goal achievement or, 
as some suggestedj they need to incorporate an examination of the unintended 
as well as intended effects or actions. Performance assessment differs to 
some extent from "management studies" which focus more on issues of produc- 
tivity and efficiency. These in turn are differentiated from "operational 
reviews" that are more concerned with accuracy of data and responsible:" Stew- 
ardship with funds and other resources. There is obviously a substantial 
overlap among these broad categories but clearly performance indicators 
are concerned with the effectiveness of our institutions and programs. 

On the second dimension, it is helpful to, recognize that assessment 
occurs in a variety of forms. There are audits , normally regular assess- 
ments of quantifiable indicators in some standardized procedure (in the past 
these have normally been associated with operational or fiduciary audits 
or management efficiency or productivity audits). There are program re- 
views in which there is a far more intensive and comprehensive review or 
programs or institutions focusing primarily on examining their real value 
or effectiveness. Audits and reviews are typically done by groups external 
to the institutions although not always. Finally, there are self studies 
or assessments done by the institutions for some form of external recog- 
nition or for its own self improvement. These different kinds of assess- 
ments or evaluations may include performance dimensions, or judgements about 
it. 

The performance assessment approach is further complicated by the fact 
that they are often done by a number of different institutions or agencies. 
For instance, in the context of this conference, examples of audits or re- 
views were reported:, a) from the executive branch of government usually by the 
budget or finance office; b) from the legislature where in the more com- 
plex states there may be legislative fiscal, audit, and- program evaluation 
staffs each serving the legislature; c) from the state higher educational 
governance system where governing borads conduct reviews to fulfill their 
managerial responsibilities for spending and allocating money while the 
coordinating board may be more interested in policy reviews to justifying 
requests to the legislature' or for planning. Naturally, many assessments 
occur within institutions themselves either by institutional and analytic 
studies offices or other units and external review by accrediting agencies 
are also part of the picture. 

This discussion of the definition of performance, the forms of assess- 
ment or evaluation and agencies involved is necessary to distinguish some of 
the vagueness in the terms and process that we use in talking about per- 
formance assessment, but it also highlights the potential^ complexity; there is a 
necessity in adopting a performance approach to obtain some agreement on 
what the terms mean, what agencies and groups are involved in each state. 
This is perhaps better accomplished with clear examples of the questions 
that are being asked about performance, the precise identification of in- 
dicators used, the clear understanding of the level of institutional pro- 
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gram being assessed, and the roles and functions of the parties who are 
involved in that process rather than general categories or abstract def- 
initions. 

C. The Guidance System: Concern for Consensus or Control Capacity? 

In his book, The Active Society , Amitai Etzoni distinguishes processes 
which social systems use to reach a consensus or make decisions from its 
capacity to control the Social system (That is to collect and analyze data 
and to implement decisions). In his argument, an active society is one 
that has a high capacity on both. In the context of this conference the 
greatest emphasis seemed to be given to the process of reaching consen- 
sus on performance criteria. Perhaps this was due to concerns about cred- 
ibility, about legislative-higher education disagreements, and about the 
general uncertainty regarding consequences and motives of other interested 
parties. In discussions about the consensus capacity witSin our states, 
great emphasis was placed on the communication gap between institutions 
and legislatures and some of the negative stereotypes that each had of 
the other were aired. The major concern was that educators and legislators 
find means of working (.together more effectively. It was pointed out that 
the anti-legislative, anti-government bureaucracy attitude in the country 
may be as strong as the anti-higher educational feeling. Therefore the two 
groups share a common problem— explaing what their efforts accomplish — 
which might be the basis for their dialogue. However, mechanisms for gen- 
erating dialogue were less clear and probably will vary by state. It was 
recognized that there is a need to involve state executive agency staff 
members as well and that there were often limitations due to the fact that 
legislative decision power was often situated outside of education commit- 
tees or staff groups and that the real criterion for policy decisions were 
often much more pragmatic and political concerns regarding constituency 
interests. 

On the control side of our higher education state guidance system the 
growth of information systems and analytic staff i& institutions of higher 
education is known. It was noted that this was now being accentuated by 
growth of such staff members in legislative and executive branches. There 
is the potential in many states that these groups may all be working in the 
same area. Because of different concerns and purposes of legislatures, 
executive, higher education agency and institutional and staff members, 
there also is the possibility that it may become a duplicative or con- 
flict situation: distrust in one anothers data and analyses or compe- 
tetion by these staff to gain greater favor with their supervisors. The lat- 
ter issue clearly raises the question of cost of all of our ccmtrol system 
apparatus for higher education. What is its real cost? What is it adding 
to our bureaucratic procedures and how might it be simplified if states 
move to performance approaches? ' What kinds of things might be replaced? 
There were no answers but it clearly is an area that needs to be explored. 

There was a general feeling on the part of conference participants 
that, at least for the moment, they are less concerned about the details 
of the control mechanisms than they were about the need to develop con- 
sensus if a performance approach were to work. 

D. Creating Consensus Around Performance Criteria * 



The need for dialogue in establishing performance criteria was previ- 
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ously identified. The problem of creating consensus performance criteria 
is separable from the problem of involvement in the evaluation process. In 
the context of the discussion groups, there seemed to be two quite different 
approaches to the process of creating consensus on performance criteria for 
a state that was considering moves in that direction. The one extreme sug- 
gested starting with legislative and executive needs for decision making 
as "valid" information and as a basis for defining the scope and nature of 
data required for performance criteria. The other approach begins with the 
assumption that valid information begins with institutions 1 statements of 
mission, role, and scope which are prerequisites for institutions develop- 
ing their own performance criterion measures which they then present to 
state government. Obviously these two approaches, no doubt, represent dif- 
ferences in styles of state coordination and governance, but they may also 
represent very different notions of performance indicators, the latter fo- 
cusing on what institutions think they accomplish and the former focusing 
on the concerns of legislators and executive, officials In their daily. po- 
litical lives- Ttere is some evidence on the differing priorities one might 
find in these two various approaches reflected in Sid Micek's paper in which 
he identifies outcomes preferred by various constituencies and these deserve 
careful review. 

Another interesting dichotomy of process was identified by the discus- ^ 
sion groups. One suggested that the process of creating performance cri- 
teria would essentially be one of consensus flowing out of informal commun- 
ication and discussions encouraged by Mr* Klebanoff . On the other hand, 
some groups talked of it in terms of institutions negotiating with the s.tate, 
again this, no doubt, reflects the differences in state governance style, 
emphasis on collective bargaining, etc., but it does point out that the 
process by which performance criteria are to be decided is likely to vary 
by state. 

a 

More specifically in the discussions there was a varied picture of the 
appropriate role and function of the state legislature, state executive 
officers, higher education coordinating officials and institutions in 
determining performance criteria. However, there was general agreement 
that if institutions or the higher education agencies didn't do something 
it was likely to be mandated and/or initiated from the state level. Also 
there was general agreement that the process through which performance cri- 
teria were established should involve open discussion. The experience of 
the states involved in performance reviews gave us some limited information^ 
on the experience of establishing performance criteria, however, the perform— fi- 
ance reviews were often quite limited in scope and this process needs 
greater attention. 

E. The Measurement Issues . 0 

The measurement of performance indicators, as Sid Micek pointed out, 
are complicated by a number of issues such as: the lack of existing ex- 
plicit measures for outcomes that are well accepted", the fact that tech- 
niques for analyzing and collecting outcome measures were not well under- 
stood, that the goals and objectives that indicators might be oriented toward 
were often not existent or immeasurable, that there is substantial fear that 
they will be misused, and that their mode of presentation is often a major 
concern. Yet in the conference itself, it became apparent" that program 
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reviews in several states are beginning to use outcome or performance cri- 
teria in serious attempts to assess programs. NCHEMS, of course, has a 
major project focused on categorizing outcome criteria which was presented 
although an operational discussion of its usefulness was not included in 
the program. It was pointed out performance measures, whether those devel- 
oped by NCHEMS, or others, should probably not be stipulated for all in- 
stitutions or states without much more careful consideration of their 
appropriateness in the specific context. 

In establishing the capacity to measure performance outcomes, a number 
of issues were highlighted: 

1. The content of each. particular performance criterion needs to be care- 
fully spelled out. What in fact is being measured? Is it an immediate 
outcome or an impact on the larger society? Can it be measured now or 
only after passage of time? The NCHEMS ^outcomes it was felt, offered 

a useful initial categorization of a variety of different content areas 
of outcomes. 

2. The level of measurement is an issue of considerable debate. Should per- 
formance measures be specified for programs or for institutions? Those 
most interested in resource allocation. and concerned about the current 
budgetary constraints seemed to favor performance outcomes focused on 
program level assessments. Others who were more concerned about issues 
of institutional autonomy and about the more abstract and/or value laden 
natureflf institutional outcomes which are very difficult to measure 

or to obtain agreement on seemed to prefer institutional level measures, 
if any. It is, however, reasonably clear in a survey done by Robert 
Barak last year that most of the existing attempts to develop perform- 
ance measures by state higher education agencies focus on programs rather 
than total institutions and this may reflect the current direction. 

3. Regardless of the content or level on which the performance criteria is 
measured, one major difficulty in using performance indicators is im- 
puting causation from the higher education experience to changes in the 
outcome indicators. The complex problems of setting up comparative re- 
search programs to control all the variables that migjit influence the out- 
comes are extremely time consuming, complex, and potentially expensive. 
The more positive view was that a longitudinal review of the same per- 
formance measures, used consistently, might be some barometer of effect- 
iveness or the impact of other changes that were occurring but that these 
had to be interpreted with caution. 

4. In iight of the difficulty of imputing causation to the performance mea- 
sures the problem of deciding what standards to use for evaluation emerges. 
One approach is to relate performance measures to goals. But this may 
overlook many important unintended consequences of a particular program 

or institutional effort which might be far more important than goal 
achievement. Another criterion is comparison. But here the difficulties 
of finding similar institutional types or programs and the difficulties 
of insuring data consistency, collection and analysis procedures in larg- 
er scale institutional : >arisons are extensive. This problem has al- 
ready been identified ± major focus for NCHEMS 1 institutional exchange 
project which unfortunately does not currently focus on outcome measures. 
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5. A final measurement issue concerns the multiplicity of performance re- 
viewers that may be interested in performance evaluation who may expect 
multiple criterion if there is limited agreement on which are most approp- 
riate- Even further, the exaggerated fear of misuse of simplistic 
indicators may lead to an overemphasis on the development of multiple 

outcome measures. This not only increases the difficulty of perform- 
ance assessment but leads to direct conflict with state-level decision 
makers' concerns that they not be inundated with too much data. 

While there are other measurement issues connected with performance 
assessment, these represent some of the most crucial that were mentioned 
during the course of this conference. 

F. The Evaluation or Assessment Process . 

The use of performance criterion in an evaluation process, even when 
the criteria are carefully agreed upon, inevitably raises questions about 
what the particular performance criteria means. While the detailed ques- 
tion of roles and function of the various constituencies in evaluation was 
addressed in the groups, a number of alternatives were implied. These 
ranged from institutional based evaluation teams, to state higher education 
agency research staff, to state legislators and executive auditing staff, 
to outside accreditation teams, and to various combinations. These are 
partially related to the purpose of the evaluation — where it is to provide 
information for improvement or to make final judgments on contribution. 
This assessment or evaluation issue is further complicated by the ques- 
tion of who is the best source of data, should it be collected and provided 
the institutions themselves, by state agencies, or by some other group? 
To the extent that there is a substantial amount of distrust, these issues 
become more and more central to making an effective assessment process work. 
While many of these issues were identified, there was no real consensus 
around the role and function that the various groups should play in the 
data analysis and assessment process. Obviously institutions desire to 
claim as large a role as possible yet most of the efforts initiated today 
appeared to be from the state level. 

G. The Linking Dilemmas; Relating Outcomes to Action 

There was ldttle experience reported by the participants in relating 
the performance measures to the budgeting function. It was suggested 
that budgets were influenced by very marginal judgments that might be made 
on the basis of information available to them through some program review 
or audit. For the most part there was a skeptical feeling that the form 
of budgeting really had very little effect on outcomes and that it was not 
likely to affect the number of dollars received in higher education. There- 
fore there was little support in the discussion groups for attempting to 
link performance measurement to budgeting. On the positive side there was 
an expression of feeling that performance parameters could be very usefully 
linked to the identification of problems, that both qualitative and quan- 
titative performance measures might be very useful to encourage the gener- 
ation of alternative means for achieving outcomes which then could be tried 
and evaluated. 
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Despite these ways In which outcome information might be linked to action, 
the general tendency was to support the development of performance indicat- 
ors first 'as some states have been doing, and only consider moving into per- 
formance budgeting after there is substantial experience with the indicators 
themselves. There seemed to be some tendency to stress the development of 
quantitative performance indicators first and some impression that these 
might be the only ones that would have any potential use in the budgeting pro- 
cess, if at all. There was a stronger suggestion that performance measures 
should not or could not be related to budgeting because budget decisions have 
to be made annually while performance indicators change more slowly and be- 
cause decisions in the budget are on the basis of inputs or efforts which 
are easily understood and that we are not likely to have strong sense of the 
linkage between outcomes and budget dollars for years to come. 

Finally, if performance measures were applied to the budgetary process, 
the technical issue raised was the old NCHEMian dilemma: there was a concern 
that there should be a close fit between the budgetary reporting structure 
and an outcome which should reflect their cause and effect association. This 
was a greater concern of those who were most interested in program-oriented 
performance measures. The plausible linking structure becomes less clear 
as one favors institutional performance budgets except to provide institutional 
lump sum appropriations. 

SUMMARY AND RECOMMENDATIONS ; 

In summary, there appeared to be no strong agreement on why performance mea- 
sures were necessary or desirable and no clear evidence as to their conse- 
quences. There was general agreement that they may be inevitable, may have 
desirable consequences, and that there is a need to pursue their develop- 
ment. This seemed to be felt more strongly by state level officials. The 
conference identified numerous recent examples or experiences with the use 
of performance indicators for program audits or review but little in the 
area of performance budgeting. There was very strong feeling that the per- 
formance criteria and performanceevaluation processes need to be the focus 
of substantial dialogue among interested parties but that the format will 
vary from state to state. The participants reflected great interest and sub- 
stantial caution ox tentativeness. Whether the .tentativeness expressed in this 
conference was a form of future shock, fear, or impressions of faddishness was 
still unclear. However, it did seem apparent that the participants were not 
ready for a "Grand Scheme 11 , but would subscribe to recommendations such as 
the following: 

If outcome or performance reviews are to have a positive effect and if 
thereis any potential in relating them to the budget process, there needs 
to be a new set of attitudes (or at least an image) on the part of insti- 
tutional based innovators, budget officers, and state level officials. 
Innovators, need to avoid the pitfall of having "answers in search of 
money". Budgeters have to focus on "outcomes and not inputs". Legisla- 
tors lieed to convince others that they are willing to "provide incent- 
ives" to give the approach an opportunity to develop. All need to treat 
this development as a "new experience"; as a theory to be tested and not 
as a "New Dogma" which conflicts with the "truth or past experience". 

2. Mechanisms for dialogue among various constituencies on a state by state 
basis may be a very useful step. Some suggestions included: 
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A. Informal institution/legislator/exccutive branch staff meetings 
to discuss issues and problems of higher education and particu- • 
larly focused around performance measures. 

B. The possible creation of an independent citizens council to collect 
and maintain information on the performance of higher education. 

C. The development of some form of interstitial group uncommitted to 
any constituency who understand education, have evaluation exper- 
ience, have* some expertise in public policy assessment, and who 
understand the legislative process. " Such a group or task force 
might guide the development of an effective performance system while 
allaying some of the fears of each constituency about the potential 
abuses by the others. 

3. There is a need to review the 'experience with performance measures and 
performance review or auditing to date. There are some preliminary ef- 
forts underway which should be referenced in the near future. However, 
there is also a need to have some more extensive research on approaches 
being used and the effects of them. 

4. There is a need to fund major development projects such as the perform- 
ance budgeting FIPSE project in the state of Tennessee. A set of pro- 
jects in different states designed to constrast the complexity of higher 
education structure in the state, the variation in forms of higher ed- 
ucation coordination mechanisms, and differing types of performance cri- 
teria would provide a set of comparative projects that might give us more 
insight into. the consequences not only of performance assessment but of 
the extent to which it could be tied to budgeting. 

5. There was support for continued meetings or conferences to explain 
this topic. The suggestions ranged from meetings of the varied con- 
stituents in states with active programs in this area, to regional 
meetings of states not yet involved, to the creation of a broadly 
representative National Task Force to set forth some principles and 
guidelines. 

This surammary started out with a Grook so let me end it with a limmerick 
which I will try my best to keep clean: 

"There once was a conference at Coronado; 
Where participants sougjht the new El Dorado; 
Performance and budgeting were the game; 
The rhetoric was both new and the same; 
But spiced with candid and classy bravado. 
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